Image capture device, image processor and image processing method

ABSTRACT

An image capture device includes: an image capturing section which captures a subject image and generates an image; an acceleration sensor which detects acceleration; an angular velocity sensor which detects angular velocity; a controller which determines, according to a result of detection obtained by the angular velocity sensor, a first correction angle that is based on a result of detection obtained by the acceleration sensor and a second correction angle that is based on a result of detection obtained by the angular velocity sensor; and an image processing section which rotates the generated image based on an angle obtained by adding together the first and second correction angles.

BACKGROUND

1. Technical Field

The present disclosure relates to an image capture device.

2. Description of the Related Art

Japanese Laid-Open Patent Publication No. 2002-94877 discloses anelectronic camera. This electronic camera writes image data representinga cropped portion of a captured image to a storage medium. And thiselectronic camera rotates the coordinates of that cropped portion of thecaptured image in such a direction in which the shake of an image on thescreen in a tilt direction can be canceled.

The present disclosure provides an image capture device that can make atilt correction more appropriately.

SUMMARY

An image capture device according to an embodiment of the presentdisclosure includes: an image capturing section configured to capture asubject image and to generate an image; an acceleration sensorconfigured to detect acceleration; an angular velocity sensor configuredto detect angular velocity; a controller configured to determine,according to a result of detection obtained by the angular velocitysensor, a first correction angle that is based on a result of detectionobtained by the acceleration sensor and a second correction angle thatis based on a result of detection obtained by the angular velocitysensor; and an image processing section configured to rotate thegenerated image based on an angle obtained by adding together the firstand second correction angles.

According to the technique of the present disclosure, an image capturedevice that can make a tilt correction more appropriately is provided.

These general and specific aspects may be implemented using a system, amethod, and a computer program, and any combination of systems, methods,and computer programs.

Additional benefits and advantages of the disclosed embodiments will beapparent from the specification and Figures. The benefits and/oradvantages may be individually provided by the various embodiments andfeatures of the specification and drawings disclosure, and need not allbe provided in order to obtain one or more of the same.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram illustrating a configuration for a digitalcamcorder 100 according to a first exemplary embodiment.

FIGS. 2A to 2C schematically illustrate the axes of detection of anacceleration sensor and an angular velocity sensor.

FIG. 3 is a flowchart showing the procedure of a tilt offset correctionoperation.

FIGS. 4A and 4B schematically illustrate how to perform the tilt offsetcorrection operation.

FIG. 5 is a flowchart showing how to calculate clip angles for tiltcorrection and rotational shake correction and how to calculate acorrection angle according to the shooting situation.

FIG. 6 schematically shows the outputs of the angular velocity sensor.

FIG. 7 schematically illustrates the clipping amplitudes to bedetermined for the tilt correction and the rotational shake correctiondepending on the shooting situation.

FIG. 8 schematically shows how the output value of the accelerationsensor changes.

FIG. 9 illustrates a configuration for an image processor 300 as asecond exemplary embodiment of the present disclosure.

DETAILED DESCRIPTION

Hereinafter, embodiments will be described in detail with reference tothe accompanying drawings as needed. It should be noted that thedescription thereof will be sometimes omitted unless it is absolutelynecessary to go into details. For example, description of a matter thatis already well known in the related art will be sometimes omitted, sowill be a redundant description of substantially the same configuration.This is done solely for the purpose of avoiding redundancies and makingthe following description of embodiments as easily understandable forthose skilled in the art as possible.

It should be noted that the present inventors provide the accompanyingdrawings and the following description to help those skilled in the artunderstand the present disclosure fully. And it is not intended that thesubject matter defined by the appended claims is limited by thosedrawings or the description.

(Embodiment 1)

1-1. Outline

The digital camcorder 100 of this embodiment has a tilt correctionfunction and a rotational shake correction function. In thisdescription, the “tilt correction function” is a function for correctingthe tilt of a captured image which is caused by the device's own tiltwith respect to the direction of the acceleration of gravity. Thedigital camcorder 100 corrects electronically the tilt of an image bycalculating, based on the output of an acceleration sensor 260, how muchits own device tilts with respect to the direction of the accelerationof gravity and by rotating the captured image in a direction in whichthe tilt of the captured image can be canceled. On the other hand, therotational shake correction function refers herein to the function ofreducing the influence of the device's own shake (i.e., so-called“camera shake”) in the roll direction on the captured image.

The digital camcorder 100 corrects electronically the captured image byrotating, based on the output of an angular velocity sensor 250, thecoordinates of a cropped portion of the captured image in such adirection in which the tilt in the roll direction and/or the rotationalshake can be canceled.

Hereinafter, the configuration and operation of the digital camcorder100 of this embodiment will be described with reference to theaccompanying drawings.

1-2. Configuration

First of all, the configuration of the digital camcorder 100 of thisembodiment will be described with reference to FIG. 1.

FIG. 1 is a block diagram illustrating a configuration for the digitalcamcorder 100. This digital camcorder 100 includes an optical system110, a lens driving section 120, a CMOS image sensor 140, an A/Dconverter 150, an image processing section 160, a buffer 170, acontroller 180, a card slot 190, a memory card 200, an operating section210, a display monitor 220, an internal memory 240, an angular velocitysensor 250, and an acceleration sensor 260. In this digital camcorder100, a subject image that has been produced through the optical system110 including a single or multiple lenses or multiple groups of lensesis sensed by the CMOS image sensor 140. The image processing section 160subjects the image data that has been generated by the CMOS image sensor140 to various kinds of processing and then writes processed image datato the memory card 200.

Hereinafter, these components of this digital camcorder 100 will bedescribed in further detail.

The optical system 110 has a diaphragm, an optical image stabilizer(OIS) lens and multiple groups of lenses including a zoom lens, and afocus lens. By moving the zoom lens along the optical axis, the subjectimage can be either zoomed in on or zoomed out. Also, by moving thefocus lens along the optical axis, the focus of the subject image can beadjusted. In FIG. 1, three lenses are illustrated. However, this is onlyan example and any other appropriate number of lenses may be used toform the optical system 110 according to the functions required.

The OIS lens is movable within a plane that crosses the optical axis ofthe optical system 110 at right angles. By shifting the OIS lens in sucha direction as to cancel the shake of the digital camcorder 100, theinfluence of the shake of the digital camcorder 100 on the capturedimage can be reduced.

The diaphragm adjusts the size of the aperture either in accordance withthe user's setting or automatically, thereby controlling the quantity oflight transmitted.

Optionally, the optical system 110 may further include a zoom actuatorwhich drives the zoom lens, an OIS actuator which drives the OIS lens, afocus actuator which drives the focus lens, and a diaphragm actuatorwhich drives the diaphragm.

The lens driving section 120 drives these various kinds of lenses anddiaphragm included in the optical system 110 by controlling the zoomactuator, focus actuator, OIS actuator and diaphragm actuator includedin the optical system 110.

The CMOS image sensor 140 detects the subject image that has beenproduced by the optical system 110, thereby generating analog imagedata. The CMOS image sensor 140 performs various kinds of operationsincluding exposure, transfer and electronic shuttering.

The A/D converter 150 converts the analog image data that has beengenerated by the CMOS image sensor 140 into digital image data.

In this embodiment, the image capturing section 270 is made up of aplurality of components including the optical system 110, the CMOS imagesensor 140 and the A/D converter 150. The image capturing section 270sequentially generates digital image data, including a plurality offrames that are continuous with each other, and outputs the digitalimage data one after another.

The image processing section 160 performs various kinds of processing onthe image data that has been generated by the CMOS image sensor 140,thereby generating image data to be displayed on the display monitor 220or image data to be written on the memory card 200. For example, theimage processing section 160 performs gamma correction, white balancecorrection, defect correction and various other kinds of processing onthe image data that has been generated by the image capturing section270. Also, the image processing section 160 compresses the image datathat has been generated by the image capturing section 270 compliantwith predetermined standard such as the H.264 standard or the MPEG-2standard. The image processing section 160 may be implemented as adigital signal processor (DSP) or a microcomputer (microprocessor), forexample.

The image processing section 160 subjects the image data to rotationprocessing, thereby reducing the influence to be caused in the rolldirection to the image produced on the CMOS image sensor 140 by thedevice's own tilt or rotational shake. Suppose a situation where theshooter has shot the subject image with the digital camcorder 100 tiltedθ degrees counterclockwise. In that case, the subject image will becaptured so as to be tilted θ degrees counterclockwise. At this time,the image processing section 160 rotates the coordinates of a portion ofthe image data to be cropped θ degrees clockwise and then crops imagedata. This cropping processing will be described in detail later. Inthis manner, the image processing section 160 generates image data thatis less affected by the device's own tilt or shake.

The controller 180 controls the overall operation of this digitalcamcorder 100. The controller 180 may be implemented as a semiconductordevice, for example. The controller 180 may be implemented as only a setof dedicated hardware components or as combination of hardwarecomponents and software. The controller 180 may be implemented as amicrocomputer, for example.

The buffer 170 functions as a work memory for the image processingsection 160 and the controller 180 and may be implemented as a DRAM or aferroelectric memory, for example.

The card slot 190 is an interface, to/from which the memory card 200 isreadily insertable and removable, and can be connected to the memorycard 200 both mechanically and electrically. The memory card 200includes a flash memory, a ferroelectric memory or any other kind ofinternal memory, and can store image files and other data that have beengenerated by the image processing section 160.

The internal memory 240 may be implemented as a flash memory or aferroelectric memory, for example, and may store a control program forcontrolling the overall operation of this digital camcorder 100.

The operating section 210 is a generic term which collectively refers tovarious kinds of user interfaces through which the user can enter his orher instructions. The operating section 210 includes cross keys, anENTER button and tilt correction setting buttons which accept the user'sinstructions.

The display monitor 220 may display either an image represented by theimage data that has been generated by the CMOS image sensor 140 (i.e., athrough-the-lens image) or an image represented by the image data thathas been read out from the memory card 200. In addition, the displaymonitor 220 can also display various kinds of menus which allow the userto change various settings of this digital camcorder 100.

Next, the respective axes of detection of the acceleration sensor 260and the angular velocity sensor 250 provided for the digital camcorder100 of this embodiment will be described with reference to FIGS. 2A to2C, each of which schematically illustrates the axes of detection of theacceleration sensor 260 and the angular velocity sensor 250.

The acceleration sensor 260 detects the tilt of this digital camcorder100 with respect to the direction of the acceleration of gravity. Asshown in FIG. 2A, the acceleration sensor 260 of this embodimentincludes a sensor which detects an acceleration component in the opticalaxis direction (i.e., the Z-axis direction) of this digital camcorder100, a sensor which detects an acceleration component within a planethat crosses the Z-axis at right angles and in the horizontal direction(i.e., X-axis direction) of this digital camcorder 100, and a sensorwhich detects an acceleration component within a plane that crosses theZ-axis at right angles and in the perpendicular direction (i.e., Y-axisdirection) of this digital camcorder 100. In this description, thesesensors will be collectively referred to herein as an “accelerationsensor 260”. It should be noted that if the optical axis of this digitalcamcorder 100 is supposed to be the Z-axis, the X- and Y-axes are fixedon this digital camcorder 100. And even if this digital camcorder 100gets tilted with respect to the direction of the acceleration ofgravity, the X-, Y- and Z-axis directions are fixed on this digitalcamcorder 100.

The acceleration sensor 260 outputs signals representing accelerationcomponents which have been detected by itself in the X-, Y- and Z-axisdirections to the controller 180. By analyzing the respective outputsignals in the X-, Y- and Z-axis directions of the acceleration sensor260, the controller 180 determines a first quantity of correction (i.e.,a first correction angle) to correct the tilt of this digital camcorder100 with respect to the direction of the acceleration of gravity. Inthis case, if the respective values of the acceleration components thathave been detected in the X-, Y- and Z-axis directions are indicated byX, Y and Z, respectively, the tilt angle θ₀ defined by this digitalcamcorder 100 with respect to the direction of the acceleration ofgravity can be calculated by the following Equation (1):

$\begin{matrix}{\theta = {\tan^{- 1}\left( \frac{X}{\sqrt{Y^{2} + Z^{2}}} \right)}} & (1)\end{matrix}$

If the digital camcorder 100 is not tilted in the pitch direction (Pdirection), the value Z of the acceleration component in the Z-axisdirection becomes equal to zero. As a result, the tilt angle θ₀ thattakes only the rotation in the roll direction (R direction) into accountis calculated by the following Equation (2) using the values of theacceleration components in the X- and Y-axis directions:

$\begin{matrix}{\theta = {\tan^{- 1}\left( \frac{X}{Y} \right)}} & (2)\end{matrix}$

For example, suppose the digital camcorder 100 defines a tilt angle of45 degrees with respect to the direction of the acceleration of gravityas shown in FIG. 2B. In that case, if the magnitude of the accelerationof gravity is 1 G, the acceleration sensor 260 detects 0.707 G as the Xand Y values. On the other hand, if the camcorder itself defines a tiltangle of 30 degrees with respect to the direction of the acceleration ofgravity as shown in FIG. 2C, then the acceleration sensor 260 detects0.500 G and 0.866 G as the X and Y values, respectively.

The angular velocity sensor 250 detects the angular velocity of thisdigital camcorder 100. As shown in FIG. 2A, the angular velocity sensor250 of this embodiment includes a sensor which detects the angularvelocity of the movement of this digital camcorder 100 to be caused inthe roll (R) direction due to a camera shake, for example. The angularvelocity sensor 250 may further include a sensor for detecting theangular velocity in the yaw direction and a sensor for detecting theangular velocity in the pitch direction, in addition to the sensor fordetecting the angular velocity in the roll direction. By analyzing theoutput signal of the angular velocity sensor 250 as for the rolldirection, the controller 180 calculates a second quantity of correction(i.e., a second correction angle) to make a correction on the shake inthe roll direction.

1-3. Tilt Correction Operation

This digital camcorder 100 has the function of making a tilt offsetcorrection in the roll direction. In addition, the digital camcorder 100also has the function of correcting the tilt and rotational shake of acaptured image in the roll direction.

Hereinafter, it will be described exactly how this digital camcorder 100makes a tilt offset correction in the roil direction and corrects thetilt and rotational shake of a captured image in the roll direction.

1-3-1. Tilt Offset Correction Operation

First of all, the tilt offset correction operation (function) will bedescribed in detail with reference to FIGS. 3, 4A and 4B. FIG. 3 is aflowchart showing the procedure of the tilt offset correction operation.FIGS. 4A and 4B schematically illustrate how to perform the tilt offsetcorrection operation.

This digital camcorder 100 has various applications, one of which is awearable camera that a person uses by putting it on his or her face orclothes as shown in FIGS. 4A and 4B. If this digital camcorder 100 isused as a wearable camera, the wearer can record, as a still picture ora moving picture, what he or she is experiencing right in front of himor her (e.g., how he or she is skiing) and the viewer can enjoy suchvideo with a lot of presence that was recorded by the wearer.

In such a situation, when the user puts on this digital camcorder 100,the digital camcorder 100 may get tilted with respect to the directionof the acceleration of gravity and the captured image may get tilted inthe roll direction depending on how he or she is wearing it as shown inFIG. 4B. For example, if he or she puts the digital camcorder 100 on hisor her face, the digital camcorder 100 may not be attached horizontallyand may get tilted with respect to the direction of the acceleration ofgravity. In that case, the shooting session will be carried out with thedigital camcorder 100 left tilted in the roll direction, thus making therecorded image uncomfortable to view.

Thus, to avoid such an unwanted situation, on accepting a predeterminedcorrection instruction, the image processing section 160 rotates theimage supplied from the image capturing section 270 in such a directionas to cancel the tilt of the image by using the reference angle θ₀(offset angle) that has been calculated based on the output signal ofthe acceleration sensor 260 as a reference of tilt correctionprocessing. Even if the digital camcorder 100 is not held horizontally,the tilted image captured by the digital camcorder 100 held in such atilted position can be corrected as if the image was shot in ahorizontal position by rotating the image using the reference angle θ₀as the reference of the tilt correction processing. As a result, even ifthe digital camcorder 100 is not held horizontally, appropriate videocan still be recorded in the horizontal position and the video that hasbeen shot in such a tilted position can still be viewed comfortably.

First of all, the user puts this digital camcorder 100 on his or herface or clothes, for example. Next, he or she turns the digitalcamcorder 100 ON, when the controller 180 instructs a power supplysection (not shown) to supply power to respective sections that formthis digital camcorder 100. In response, the CMOS image sensor 140starts capturing an image and the image processing section 160 startsimage processing. Meanwhile, the acceleration sensor 260 startsdetecting acceleration components in the X-, Y- and Z-axis directionsand the controller 180 analyzes the result of detection of theacceleration in the three axis directions, thereby sensing how much thedigital camcorder 100 is tilted with respect to the direction of theacceleration of gravity.

The controller 180 monitors if the controller 180 has received anyinstruction to make a tilt correction from the user who has put thedigital camcorder 100 on (in Step S200). Optionally, after the digitalcamcorder 100 has been turned ON, the controller 180 may issue such aninstruction to make a tilt correction by itself. Alternatively, when theuser presses down a movie recording button (i.e., when he or she issuesan instruction to record a moving picture) or taps a tilt correctionsetting button included in the operating section 210, the controller 180may issue an instruction to make a tilt correction. Still alternatively,the tilt correction instruction may even be issued by a remote devicesuch as a smart phone. In this embodiment, by operating the operatingsection 210, the tilt correction instruction is given to the controller180.

Next, if the controller 180 has accepted such a tilt correctioninstruction from the user through the operating section 210 (i.e., ifthe answer to the query of the processing step S200 is YES), then thecontroller 180 analyzes the output of the acceleration sensor 260 (inStep S210). And at the timing to make the tilt correction as specifiedby the user, the controller 180 performs the arithmetic operation givenby Equation (1) based on the output value of the acceleration sensor260. Then, the controller 180 determines the tilt angle θ₀ of thedigital camcorder 100 in the roll direction with respect to thedirection of the acceleration of gravity (in Step S210).

In this embodiment, the controller 180 is supposed to determine the tiltangle θ₀ of the digital camcorder 100 in the roll direction with respectto the direction of the acceleration of, gravity based on the respectiveacceleration components in the three axis directions that have beenobtained by Equation (1). However, this is just an example of thepresent disclosure. Alternatively, if the digital camcorder 100 is nottilted in the pitch direction (P direction), the controller 180 may alsodetermine the tilt angle θ₀ based on the respective accelerationcomponents in the two axis directions that have been obtained byEquation (2). That is to say, the controller 180 can calculate the tiltangle θ₀ based on the respective acceleration components in at least twoout of the three axis directions.

Next, the controller 180 sets the tilt angle θ₀ to be an angle −θ₀ thatcancels the tilt of the video (i.e., an offset angle −θ₀) in Step S220.

Subsequently, the controller 180 notifies the image processing section160 of this offset angle −θ₀ (in Step S230).

Finally, the image processing section 160 rotates the image that hasbeen generated by the image capturing section 270 by defining the tiltangle θ₀ to be the reference of the tilt correction processing.

In this description, “to rotate an image by defining the tilt angle θ₀to be the reference of tilt correction processing” means the following.As described above, the controller 180 accepts a tilt correctioninstruction and determines the device's own tilt angle θ₀ in the rolldirection with respect to the direction of the acceleration of gravity.

-   -   (1) if the device's own tilt angle with respect to the direction        of the acceleration of gravity remains θ₀ even after the tilt        angle has been determined, then the image processing section 160        rotates, by that tilt correction angle θ₀, the image that has        been generated by the image capturing section 270; but    -   (2) if the device's own tilt angle has increased by θ₀₁ from the        tilt angle θ₀ after the tilt angle has been determined, then the        controller 180 calculates that extra tilt angle θ₀₁ with respect        to the tilt angle θ₀ based on the result of detection obtained        by the acceleration sensor 260. And the image processing section        160 rotates, by that tilt correction angle θ₀₁, the image that        has been generated by the image capturing section 270 with the        tilt angle θ₀ defined to be the reference.

By performing this series of processing steps, the image processingsection 160 can output a tilt-corrected image and video with a reducedtilt can be recorded with good stability.

Now take a look at FIG. 4A, which illustrates respective pixel areas ofthe CMOS image sensor 140. The image processing section 160 rotates thecaptured image by the offset angle −θ₀ in the roll direction, and thenperforms image cropping processing in the effective pixel area of theCMOS image sensor 140. Then, the image processing section 160 outputsthe cropped image (i.e., the image in the valid pixel area indicated bythe solid rectangle). As a result, even if the tilt of the capturedimage in the roll direction is corrected, an image that does not includeany pixel in the optical black area can also be recorded just asintended.

The controller 180 may generate a vertical sync signal at 60 fps, forexample. And the controller 180 performs the series of processing stepsS200 through S230 within a vertical sync signal interval of 60 fps. As aresult, an appropriate tilt correction can be made in real time.

1-3-2. How to Prioritize Either Tilt Correction or Rotational ShakeCorrection According to Shooting Situation

Next, it will be described with reference to the flowchart shown in FIG.5 how to prioritize either tilt correction or rotational shakecorrection according to the shooting situation.

The digital camcorder 100 senses the shooting situation based on theresult of detection obtained by the angular velocity sensor 250 andselectively carries out either a tilt correction or a rotational shakecorrection according to the shooting situation. Specifically, thecontroller 180 determines, according to the shooting situation, whichone of these two types of correction (i.e., the tilt correction or therotational shake correction) should be given priority over the other.

Hereinafter, it will be described exactly how the controller 160 decidespriorities in its processing.

First of all, if the digital camcorder 100 is operating in the shootingmode, the controller 180 senses the shooting situation based on theresult of detection obtained by the angular velocity sensor 250 (in StepS300).

Now, it will be described with reference to FIG. 6 exactly how thecontroller 180 senses the shooting situation in Step S300. FIG. 6schematically shows the outputs of the angular velocity sensor 250 inrespective shooting situations. In FIG. 6, the ordinate represents theoutput value (deg/sec) of the angular velocity sensor 250 and theabscissa represents the time.

On the time axis shown in FIG. 6, Period A indicates a situation wherethe digital camcorder 100 is fixed on a tripod, for example (which willbe referred to herein as “Situation A”). Period B indicates a situationwhere the user is shooting video with the digital camcorder 100 held andfixed in his or her hand (which will be referred to herein as “SituationB”). And Period C indicates a situation where the user is shooting videowhile walking and allowing the digital camcorder 100 to movesignificantly (which will be referred to herein as “Situation C”). Thecontroller 180 senses the shooting situation by the amplitude orfrequency of the signal supplied from the angular velocity sensor 250.Specifically, for that purpose, the controller 180 refers to the firstand second threshold values shown in FIG. 6. The first and secondthreshold values may be stored in advance in the internal memory 240,for example. The first threshold value is used to determine whether ornot the digital camcorder 100 is in Situation A. The second thresholdvalue is used to determine whether the digital camcorder 100 is inSituation B or in Situation C. If the output value of the angularvelocity sensor 250 turns out to be equal to or smaller than the firstthreshold value, the decision is made by the controller 180 that thedigital camcorder 100 is now fixed on a tripod (in Situation A). On theother hand, if the output value of the angular velocity sensor 250 turnsout to be greater than the first threshold value but less than thesecond threshold value, the decision is made that the digital camcorder100 is now held and fixed in hand (in Situation B). And if the outputvalue of the angular velocity sensor 250 turns out to be equal to orgreater than the second threshold value, then the decision is made bythe controller 180 that the user is shooting video with this digitalcamcorder 100 while walking (in Situation C).

In this example, the shooting situation is determined to be one of threestages by providing first and second threshold values. However, this isonly an example of the present disclosure. Alternatively, the shootingsituation may also be determined to be one of two stages with only onethreshold value provided. Still alternatively, the shooting situationmay also be sensed in multiple stages with three or more thresholdvalues provided.

Next, in Step S310, the controller 180 determines the clippingamplitudes for the tilt correction and the rotational shake correctionaccording to the shooting situation that has been sensed in the previousprocessing step S300.

Hereinafter, the clipping amplitudes for the tilt correction and therotational shake correction will be described with reference to FIG. 7,which schematically illustrates the clipping amplitudes to be determinedfor the tilt correction and the rotational shake correction according tothe shooting situation.

In this embodiment, the maximum quantity of correction (i.e., themaximum correction angle) in the roll direction around the optical axis,which is obtained by adding together the clipping amplitude for tiltcorrection (which will be referred to herein as a “first upper limitangle”) and the clipping amplitude for rotational shake correction(which will be referred to herein as a “second upper limit angle”), issupposed to be X (3 degrees) as shown in FIG. 7. In this case, themaximum correction angle refers to the maximum permissible angle bywhich an image can be possibly rotated.

In this example, the maximum correction angle is supposed to be 3degrees. However, this is just an example of the present disclosure.Rather, the maximum correction angle may be changed appropriatelyaccording to the specification of a given product. The maximumcorrection angle mainly depends on the size of the effective pixel areaof the CMOS image sensor 140 described above. In this case, the largerthe effective pixel area, the larger the maximum correction angle canbe.

As shown in FIG. 7, the first and second upper limit angles are supposedto be “a” and “b”, respectively, in Period A, “c” and “d”, respectively,in Period B, and “e” and “f”, respectively, in Period C. In any shootingsituation, the sum of the first and second upper limit angles satisfiesthe relation X (3 deg.)=a+b=c+d=e+f.

In Period A, as the digital camcorder 100 is fixed on a tripod, tilt isproduced steadily but no rotational shake is generated. That is why thecontroller 180 sets the first upper limit angle a to be larger than thesecond upper limit angle b. Meanwhile, in Period C, the user is shootingvideo while walking, and therefore, acceleration other than theacceleration of gravity could be produced as the digital camcorder 100moves.

Next, the output values of the acceleration sensor 260 in respectiveshooting situations will be described with reference to FIG. 8, whichschematically shows how the output value of the acceleration sensor 260changes. In FIG. 8, the ordinate represents the output value of theacceleration sensor 260 and the abscissa represents the time. As shownin FIG. 8, as the shooting situation changes, acceleration other thanthe acceleration of gravity (i.e., acceleration to be produced as thedigital camcorder 100 moves) increases and the output signal of theacceleration sensor 260 comes to have increasing amplitude. As a result,the result of detection obtained by the acceleration sensor 260 comes tohave a lower degree of reliability.

Now take a look at FIG. 7 again. In a situation where the user isshooting while walking in Period C, the tilt correction and rotationalshake correction can be made effectively by setting the second upperlimit angle f to be larger than the first upper limit angle e. On theother hand, in Period B, the first and second upper limit angles c and dare set to have an appropriate ratio in order to make the tiltcorrection and rotational shake correction effectively.

The controller 180 changes the ratio of the clipping amplitudescontinuously so as to prevent the image from becoming discontinuousbetween frames due to a change in clipping amplitude.

In the internal memory 240, stored in advance is information about acurve representing how the ratio of the first and second upper limitangles changes according to the shooting situation such as the one shownin FIG. 7. By reference to that information about such a curve as storedin the internal memory 240, the controller 180 determines the clippingamplitudes (i.e., the first and second upper limit angles) so as toprevent the image from becoming discontinuous between frames.

Suppose the shooting situation at a certain point in time is “handheldand fixed” (Period B). At that time, the ratio of the first and secondupper limit angles becomes c to d. If the shooting situation changesinto “shooting while walking” at the next point in time while the ratiois set to be c to d, the controller 180 does not change the ratio of thefirst and second upper limit angles into e to f immediately at thatpoint in time. Instead, by reference to the information about the curverepresenting how the ratio changes as stored in the memory, thecontroller 180 changes the ratio of the first and second upper limitangles from c to d into e to f continuously. If the shooting situationremains “shooting while walking” and if the ratio being changedcontinuously becomes e to f, then the controller 180 fixes the ratio ate to f until the shooting situation changes next time.

On the other hand, if the shooting situation changes while the ratio isbeing changed continuously from c to d into e to f, then the controller180 varies the ratio being changed continuously according to theshooting situation.

In this manner, the controller 180 changes the ratio of the clippingamplitudes continuously so as to prevent the image from becomingdiscontinuous between frames by changing the clipping amplitudes.

In the example described above, the shooting situation is supposed tochange in three stages in the order of “fixed on tripod”, “handheld andfixed” and “shooting while walking”. However, this is just an exampleand the shooting situation does not have to change in these three stagesbut may further include “shooting while running”. In this manner, themodes of processing can be changed appropriately between tilt correctionand rotational shake correction according to the shooting situation.

Now take a look at FIG. 5 again.

Next, the controller 180 calculates a tilt correction angle θ₁ (i.e.,first calculated angle) based on a result of detection obtained by theacceleration sensor 260 and also calculates a rotational shakecorrection angle θ₂ (i.e., second calculated angle) based on theintegral of the outputs of the angular velocity sensor 250 (in StepS320).

Then, the controller 180 compares the first and second upper limitangles that have been obtained in Step S310 to the tilt correction angleθ₁ and rotational shake correction angle θ₂, respectively, therebydetermining the final correction angle θ₃ with the tilt correction angleθ₁ and rotational shake correction angle θ₂ clipped as needed (in StepS330).

Hereinafter, a specific situation where the ratio of the first andsecond upper limit angles is c to d will be described.

If the tilt correction angle θ₁ calculated in Step S320 is smaller thanthe first upper limit angle c, the final correction angle about the tiltcorrection (i.e., the first correction angle) is determined to be θ₁. Onthe other hand, if the tilt correction angle θ₁ is greater than thefirst upper limit angle c, the final correction angle about the tiltcorrection (i.e., the first correction angle) is clipped at, and set tobe equal to, the first upper limit angle c.

In the same way, if the rotational shake correction angle θ₂ calculatedin Step S320 is smaller than the second upper limit angle d, the finalcorrection angle about the rotational shake correction is determined tobe θ₂. On the other hand, if the rotational shake correction angle θ₂ isgreater than the second upper limit angle d, the final correction angleabout the rotational shake correction (i.e., the second correctionangle) is clipped at, and set to be equal to, the second upper limitangle d.

Then, the controller 180 calculates the final correction angle θ₃ byadding together the first and second correction angles that have beendetermined by reference to the first and second upper limit angles,respectively.

Subsequently, in Step S340, the controller 180 notifies the imageprocessing section 160 of the final correction angle θ₃ that has beendetermined in Step S330.

Finally, the image processing section 160 rotates the image that hasbeen generated by the image capturing section 270 by the finalcorrection angle θ₃.

As described above, the controller 180 senses the shooting situation byeither the amplitude or frequency of a signal supplied from the angularvelocity sensor 250. By performing this series of processing steps S300through S340, while the shooting is carried out with the camcorder fixed(i.e., in Period A), the tilt correction is prioritized so as to reducethe influence of the tilt of the digital camcorder 100 on the image moreeffectively. On the other hand, while the shooting is carried out by theuser who is walking (in Period C), the rotational shake correction isprioritized so as to reduce the influence of the rotational shake on theimage more effectively.

In this manner, either the tilt correction or the rotational shakecorrection is prioritized according to the amplitude and frequency of anoutput signal of the angular velocity sensor 250 (i.e., a shootingsituation) within an angular range that is defined by the maximumcorrection angle in the roll direction around the optical axis. As aresult, video can be recorded with good stability with its tilt reduced.

The controller 180 may generate a vertical sync signal at 60 fps, forexample. And the controller 180 performs the series of processing stepsS300 through S340 within a vertical sync signal interval of 60 fps. As aresult, either tilt correction or rotational shake correction can becarried out appropriately in real time according to the shootingsituation.

In this embodiment, the tilt offset correction operation and the tiltcorrection and rotational shake correction operations may be performedas a series of operations. In that case, the image processing section160 rotates the image by the final correction angle θ₃ using thereference angle θ₀ as a reference for tilt correction processing.

1-4. Effects

As described above, the digital camcorder 100 of this embodimentincludes: an image capturing section 270 configured to capture a subjectimage and to generate an image; an acceleration sensor 260 configured todetect acceleration; an angular velocity sensor 250 configured to detectangular velocity; a controller 180 configured to determine, according toa result of detection obtained by the angular velocity sensor 250, afirst correction angle that is based on a result of detection obtainedby the acceleration sensor 260 and a second correction angle that isbased on a result of detection obtained by the angular velocity sensor250; and an image processing section 160 configured to rotate thegenerated image based on a correction angle θ3 that is the sum of thefirst and second correction angles.

As a result, either the tilt correction or the rotational shakecorrection is prioritized according to the amplitude and frequency of anoutput signal of the angular velocity sensor 250 (i.e., a shootingsituation) within an angular range that is defined by the maximumcorrection angle in the roll direction around the optical axis, andvideo can be recorded with good stability with its tilt reduced.

In one embodiment, the controller 180 determines first and second upperlimit angles, which define the upper limits of the first and secondcorrection angles, respectively, according to either the amplitude orfrequency of an output signal of the angular velocity sensor 250.

In another embodiment, the controller 180 determines the firstcorrection angle by comparing a first calculated angle that has beencalculated based on the result of detection obtained by the accelerationsensor 260 to the first upper limit angle and also determines the secondcorrection angle by comparing a second calculated angle that has beencalculated based on the result of detection obtained by the angularvelocity sensor 250 to the second upper limit angle.

In this case, the controller 180 sets the first upper limit angle to bethe first correction angle if the first calculated angle is greater thanthe first upper limit angle and sets the first calculated angle to bethe first correction angle if the first calculated angle is equal to orsmaller than the first upper limit angle.

Also, the controller 180 sets the second upper limit angle to be thesecond correction angle if the second calculated angle is greater thanthe second upper limit angle and sets the second calculated angle to bethe second correction angle if the second calculated angle is equal toor smaller than the second upper limit angle.

(Embodiment 2)

Next, a second embodiment of the present disclosure will be describedwith reference to FIG. 9.

2-1. Configuration

FIG. 9 illustrates a configuration for an image processor 300 as asecond embodiment of the present disclosure.

The image processor 300 of this embodiment includes an image processingsection 310 and a controller 320. The image processor 300 includes anexternal interface, through which a buffer 170 may be connected to thisimage processor 300.

The image capturing section 290 includes an optical system 110, a CMOSimage sensor 140, an A/D converter 150, an angular velocity sensor 250and an acceleration sensor 260. The image processor 300 subjects theimage data that has been generated by the image capturing section 290 tovarious kinds of processing.

The A/D converter 150 of the image capturing section 290 is electricallyconnected to the image processing section 310 of the image processor300. The image processing section 310 may have the same configuration asthe image processing section 160 of the first embodiment describedabove, and a detailed description thereof will be omitted herein.

The angular velocity sensor 250 and acceleration sensor 260 of the imagecapturing section 290 are electrically connected to the controller 320of the image processor 300. The controller 320 may have the sameconfiguration as the controller 180 of the first embodiment describedabove, and a detailed description thereof will be omitted herein.

2-2. Operation

The image processing section 310 and controller 320 of this secondembodiment operate in the same way as their counterparts 160 and 180 ofthe first embodiment. Thus, this image processor 300 can also performthe tilt correction operation shown in FIGS. 3 and 5. Each of thoseoperations of this embodiment is the same as what has already beendescribed for the first embodiment, and a detailed description thereofwill be omitted herein.

2-3. Effects

As described above, an image processor 300 according to this embodimentprocesses an image supplied from an image capturing section 290including an acceleration sensor 260 that detects acceleration and anangular velocity sensor 250 that detects angular velocity. The imageprocessor 300 includes: a controller 320 configured to determine,according to a result of detection obtained by the angular velocitysensor 250, a first correction angle that is based on a result ofdetection obtained by the acceleration sensor 260 and a secondcorrection angle that is based on a result of detection obtained by theangular velocity sensor 250; and an image processing section 310configured to rotate the generated image based on a correction angle θ₃obtained by adding together the first and second correction angles.

As a result, either tilt correction or rotational shake correction isprioritized appropriately according to the shooting situation within anangular range that is defined by the maximum correction angle in theroll direction around the optical axis, and video can be shot with goodstability with its tilt reduced.

(Other Embodiments)

Although Embodiments 1 and 2 have been described herein as just examplesof the technique of the present disclosure, various modifications,replacements, additions or omissions can be readily made on thoseembodiments as needed and the present disclosure is intended to coverall of those variations. Also, a new embodiment can also be created bycombining respective elements that have been described for thoseembodiments disclosed herein.

Thus, some of those other embodiments will be described just as anexample.

Even though the digital camcorder 100 of the first embodiment describedabove includes an angular velocity sensor 250, a digital camcorder 100as a modified example of the first embodiment includes no angularvelocity sensor 250 but other than that, has the same configuration asthe digital camcorder 100 of the first embodiment. The digital camcorder100 as a modified example of the first embodiment includes no angularvelocity sensor 250, and therefore, does not prioritize either tiltcorrection or rotational shake correction according to the shootingsituation unlike the first embodiment, but does include an accelerationsensor 260 and can perform a tilt offset correction operation.

Although the digital camcorder 100 as a modified example of the firstembodiment is supposed to include no angular velocity sensor 250, thedigital camcorder 100 may also include both an angular velocity sensor250 and an acceleration sensor 260 as well. In that case, the digitalcamcorder 100 does include both of these sensors, but may still beconfigured to perform the tilt offset correction operation even withoutprioritizing either tilt correction or rotational shake correctionaccording to the shooting situation.

Furthermore, the technique of the present disclosure is also applicableto a software program defining the tilt offset correction operation andthe processing of prioritizing either tilt correction or rotationalshake correction according to the shooting situation. The operationdefined by such a program may be performed as shown in FIG. 3 or 5, forexample. Such a program may be either distributed by being stored on aremovable storage medium or downloaded over telecommunications lines.Various kinds of operations that have been described for the embodimentsof the present disclosure can be performed by making a processor builtin a computer execute such a program.

Various embodiments have been described as examples of the technique ofthe present disclosure by providing the accompanying drawings and adetailed description for that purpose.

That is why the elements illustrated on those drawings and/or mentionedin the foregoing description include not only essential elements thatneed to be used to overcome the problems described above but also otherinessential elements that do not have to be used to overcome thoseproblems but are just mentioned or illustrated to give an example of thetechnique of the present disclosure. Therefore, please do not make asuperficial decision that those inessential additional elements areindispensable ones simply because they are illustrated or mentioned onthe drawings or the description.

Also, the embodiments disclosed herein are just an example of thetechnique of the present disclosure, and therefore, can be subjected tovarious modifications, replacements, additions or omissions as long asthose variations fall within the scope of the present disclosure asdefined by the appended claims and can be called equivalents.

The present disclosure is applicable to not only digital camcorders butalso digital cameras, cellphones with camera, smart phones with camera,and various other electronic devices as well.

While the present invention has been described with respect to preferredembodiments thereof, it will be apparent to those skilled in the artthat the disclosed invention may be modified in numerous ways and mayassume many embodiments other than those specifically described above.Accordingly, it is intended by the appended claims to cover allmodifications of the invention that fall within the true spirit andscope of the invention.

This application is based on Japanese Patent Applications No.2012-193780 filed on Sep. 4, 2012 and No. 2013-052266 filed on Mar. 14,2013, the entire contents of which are hereby incorporated by reference.

What is claimed is:
 1. An image capture device, comprising: an imagecapturing section configured to capture a subject image and to generatean image; an acceleration sensor configured to detect acceleration; anangular velocity sensor configured to detect angular velocity; acontroller configured to determine one shooting situation from aplurality of shooting situations according to a result of detectionobtained by the angular velocity sensor and to determine, according tothe determined shooting situation, a first correction angle that isbased on a result of detection obtained by the acceleration sensor and asecond correction angle that is based on a result of detection obtainedby the angular velocity sensor; and an image processing sectionconfigured to rotate the generated image based on an angle obtained byadding together the first and second correction angles.
 2. The imagecapture device of claim 1, wherein the controller determines first andsecond upper limit angles, which define the upper limits of the firstand second correction angles, respectively, according to either theamplitude or frequency of an output signal of the angular velocitysensor.
 3. The image capture device of claim 2, wherein the controllerdetermines the first correction angle by comparing a first calculatedangle that has been calculated based on the result of detection obtainedby the acceleration sensor to the first upper limit angle and alsodetermines the second correction angle by comparing a second calculatedangle that has been calculated based on the result of detection obtainedby the angular velocity sensor to the second upper limit angle.
 4. Theimage capture device of claim 3, wherein the controller sets the firstupper limit angle to be the first correction angle if the firstcalculated angle is greater than the first upper limit angle and setsthe first calculated angle to be the first correction angle if the firstcalculated angle is equal to or smaller than the first upper limitangle.
 5. The image capture device of claim 3, wherein the controllersets the second upper limit angle to be the second correction angle ifthe second calculated angle is greater than the second upper limit angleand sets the second calculated angle to be the second correction angleif the second calculated angle is equal to or smaller than the secondupper limit angle.
 6. An image processor configured to process an imagesupplied from an image capturing section including an accelerationsensor that detects acceleration and an angular velocity sensor thatdetects angular velocity, the image processor comprising: a controllerconfigured to determine one shooting situation from a plurality ofshooting situations according to a result of detection obtained by theangular velocity sensor and to determine, according to the determinedshooting situation, a first correction angle that is based on a resultof detection obtained by the acceleration sensor and a second correctionangle that is based on a result of detection obtained by the angularvelocity sensor; and an image processing section configured to rotatethe generated image based on an angle obtained by adding together thefirst and second correction angles.
 7. An image processing method forprocessing an image supplied from an image capturing section whichincludes an acceleration sensor configured to detect acceleration and anangular velocity sensor configured to detect angular velocity, the imageprocessing method comprising the steps of: determining one shootingsituation from a plurality of shooting situations according to a resultof detection obtained by the angular velocity sensor; determining,according to the determined shooting situation, a first correction anglethat is based on a result of detection obtained by the accelerationsensor and a second correction angle that is based on a result ofdetection obtained by the angular velocity sensor; and rotating thegenerated image based on an angle obtained by adding together the firstand second correction angles.